Webviz: An Open Source Platform for Robotics Data Visualization
How Cruise visualizes autonomous vehicle data in the browser
On a typical drive around San Francisco, Cruise’s autonomous vehicles (AVs) make thousands of complex decisions every second. The AVs base these decisions on what they perceive via their cameras, microphones, and radar and LIDAR sensors, capturing all this data in binary format for later analysis. Our software and test engineers then explore this data, using what they learn to improve the safety and performance of our autonomous fleet.
Over the past year, we’ve developed a specialized internal tool to understand the petabytes of data our vehicles generate every month — both on the road and in simulation tests. It has helped us gain valuable insight into our data: via logs and camera feeds, 2D plots and 3D scenes, and much more.
Today, we’re excited to announce that we’re open sourcing this data visualization web application, which we’ve named Webviz. Now, anyone can drag and drop any ROS bag file into Webviz to get immediate visual insight into their robotics data.
Why we built Webviz
In the early days of Cruise, engineers who wanted to visualize AV data used a suite of tools provided by the ROS open source community. While these tools were powerful, using them required installing a full development environment on a specific platform, and manually positioning windows running separate tools on the screen — e.g., one window for reading log messages, another for drawing graphs, and another for viewing camera images. Engineers could customize their setup for a specific workflow, but replicating the same setup on a teammate’s machine was a time-consuming process.
At a Cruise hackathon, a group of engineers saw an opportunity to streamline this workflow by presenting data in a more cohesive way, making layout customization much easier and maintaining backwards compatibility. They combined the original open source tools into a single web-based application, dubbed “Webviz”. Their prototype was met with overwhelming excitement — this new workflow would dramatically lower the barrier to entry for Cruisers to explore and understand our AV data.
What started as an experimental project eventually grew into its own team, and Webviz is now one of the most widely used data analysis tools at Cruise. By streamlining the process by which we understand how our AVs see the world, interpret data, and make decisions on the road, Webviz has accelerated progress towards our mission — building the world’s most advanced self-driving vehicles to safely connect people with the places, things and experiences they care about.
How we built Webviz
At its foundation, Webviz is a web application that lets users configure different layouts of panels. Each panel is a data exploration tool, displaying information like text logs, 2D charts, and 3D depictions of the AV’s environment.
Our initial focus was to build a suite of these panels that corresponded to existing open source tools like rviz, rqt console, rqt_runtime_monitor, rostopic echo, and rqt_plot.
With further development, we added custom panels outside the realm of existing tools to serve Cruise-specific needs. With our continued customizations and gradual migration away from legacy tools, these panels became more finely tuned to help our engineers solve their problems, while staying useful for people in the open source robotics community.
Through it all, our guiding philosophy has been to provide users with a flexible set of tools that they can configure, extend, and customize for a variety of unique workflows.
How Cruise uses Webviz today
With its configurable interface, Webviz is currently supporting close to a thousand monthly active users spanning dozens of Cruise teams, all with vastly different problems and workflows. As our user base has grown to include AV engineers calibrating LIDAR sensors, machine learning engineers verifying model outputs, and QA engineers debugging test rides, Webviz has become increasingly feature-rich without compromising its flexibility.
This flexibility has empowered any Cruise team to contribute to the internal Webviz platform; after all, extending the project to handle a team-specific problem is as straightforward as creating a new panel. Internal Webviz users are also able to save panel configurations, and to easily share the parameters needed for a particular data exploration workflow among teammates. We’ve even integrated other internal tools with Webviz: our engineers can watch simulations as they run, or stream historical data directly from our internal data lake.
Earlier this year, we announced our decision to open source Worldview, a library born out of Cruise’s need to translate the data-rich worlds our cars navigate into clear and intuitive 3D scenes. Even before that, we had also developed and open sourced rosbag.js, a JavaScript library for reading ROS bag files. Both open source projects were actually developed throughout the process of creating Webviz.
As we built out Webviz to support many teams’ different workflows, we found that many of its benefits were not specific to Cruise, resulting in our decision to open source it as a general robotics data inspection tool. For this initial release, we settled on a suite of general panels that any robotics developer can leverage to explore their own data, with minimal setup. We hope the robotics community can use Webviz to enjoy this out-of-the-box data exploration and insight.
What’s next?
As we continue developing Webviz for both open source and internal use, we plan to add more panels to our open source offering and create a public API for other developers to build new panels encapsulating custom functionality.
To see Webviz in action, check out our live demo here. If you’re interested in exploring your own robotics data, you can drag your bag files into Webviz directly.
At Cruise, we are reminded daily that much of our technology could not have been built without the help of countless invaluable open source resources. We hope our own contributions will help others in the community build the next generation of visionary technology.
If you’d like to be a part of Cruise’s Engineering Productivity team, we are always looking for new teammates to help us solve problems that have never been solved before. We look forward to sharing more with you in the future!
Written in collaboration: Jacob Bandes-Storch, Brian Carlson, Robert Gifford, Audrey Li, Jan Paul Posma
